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Abstract 

MicroRNAs (miRNAs) are small RNA molecules composed of 19-22 
nt playing important regulatory roles in post-transcriptional gene regula- 
tion by inhibiting the translation of the mRNA into proteins or otherwise 
cleaving the target mRNA. Inferring miRNA targets provides useful in- 
formation for understanding the roles of miRNA involving in biological 
processes which may result in diagnosing complex diseases. Statistical 
methodologies of point estimates such as the LASSO algorithm have been 
proposed to identify the interactions of miRNA and mRNA based on se- 
quence and expression data. In this paper, we propose Bayesian LASSO 
and non-negative Bayesian LASSO to analyze the interactions between 
miRNA and mRNA using the expression data. The proposed Bayesian 
methods explore the posterior distributions for those parameters required 
in the model depicting the miRNA-mRNA interactions. For comparison 
purposes, we applied the Least Square Regression (LSR), Ridge Regres- 
sion (RR), LASSO, non-negative LASSO (nLASSO), and the Bayesian 
approaches to four public data sets which have the known interaction 
pairs of miRNA and mRNA. Comparing to the point estimate algorithms, 
the Bayesian methods are able to infer more known interactions and are 
more meaningful to provide credible intervals to take into account the 
uncertainty of the interactions of miRNA and mRNA. The Bayesian ap- 
proaches are useful for graphing the inferred effects of the miRNAs on 
the targets by plotting the posterior distributions of those parameters, 
and while the point estimate algorithm only provides a single estimate for 
those parameters. 



1 Introduction 



Mature miRNAs, which are a large family of regulatory RNAs expressed in 
animals and plants, are processed from primary transcripts in two steps by en- 
zymes of the RNase III endonucleases Drosha in the nucleus and Dicer in the 
cytoplasm (Lee et al. 2003 Hutvagner 2005). In cytoplasm, miRNA is then 



separated into two single strands by helicase enzymes and then one strand is 
combined with an Argonaute protein to form the RNA-induced silencing com- 
plex (RISC), which then mediates to regulate gene expression by base pairing 
imperfectly to the 3 '-untranslated region of target mRNAs to inhibit protein 



synthesis (Carthew and Sontheimer 2009). 



Recent researches have revealed that there exists more than 1400 human 



miRNA sequences (Kozomara and Griffths- Jones 2011 ) and it has been reported 



that miRNA plays important roles in various regulation processes including 
proliferation, apoptosis, differentiation and development, cellular identity and 



pathogen-host interactions ( Parker and Sheth 2007 Pillai et al. 2007 Carthew 



and Sontheimer 2009 ) . Given the measured sequences of miRNA and mRNA, 
an important task is to identify the regulation interactions between them. Ex- 
perimental and sequence-based methods have been proposed to predict targets 
of miRNA regulations. Examples of those algorithms include miRanda (Betel 



et al. 2008), TargetScan (Lewis et al. 2005), miRBase ( Griffiths- Jones et al 



2008) and PicTar (Krek et al. 2005). These algorithms are mainly based on 
the experimentally determined rules of miRNA regulation. The most general 
feature of miRNA regulation is that the 3'-untranslated region of target mR- 



NAs is complementary to the seed region (nucleotides 2-7) of the miRNA (Lewis 
et al. 2003). It has been shown that computational approaches for combining 



miRNA and mRNA high-throughout expression data with sequence-base puta- 
tive predictions can greatly reduce the number of targets of miRNA regulation 



( Muniategui et al. 2012b) 



Various computational algorithms have been proposed to identify the mRNA 
and miRNA interactions using high-throughout expression data sets. These al- 
gorithms include correlation, mutual information, mutiple linear regression, par- 



tial least squares, LASSO ( |Tibshirani[|1996[ ), non-negative LASSO (TaLASSO), 
variational approximations for linear regression models (GenMiR-|— HCTar- 
get, and Bayesian graphical approaches, etc. Muniategui et al. (2012b) provides 
a review for these algorithms. Among those proposed methods, GenMiR-|— |- 



( Huang et al. 2007), HCTarget (Su et al. 2011) and the Bayesian graphi- 
cal approaches dStingo et al. 2010) are probabilistic approaches. HCTarget 



and Bayesian graphical method employ the MCMC schemes for sampling the 
posterior densities for the required parameters, and while the GenMiR+-|- em- 
ploys the variational approach to approximate the densities of the parameters. 



The Bayesian method of, Stingo et al. (2010) is a rather different approach for 



determining the interactions of mRNA and miRNA, comparing to the linear 
regression models. Both the LASSO algorithm and the non-negative LASSO 
(nLASSO) algorithm have been applied to the linear regression models for mod- 



elling the miRNA-mRNA interactions (Lu et al. 2011 Muniategui et al. 2012a) 



In this paper, we propose to employ the Bayesian LASSO and non-negative 
Bayesian LASSO for evaluating the miRNA regulations. The Bayesian LASSO 
algorithm is similar to HCTarget and GenMiR-|--|- where no non-negative con- 
strains are put on the linear regression model parameters. We proposed the 



non-negative Bayesian LASSO algorithm based on the TaLASSO (Muniategui 



et al. 2012a) where the non-negative constrains on the parameters have been 



shown to be more meaningful in searching for the down regulation effects of 
miRNA on the mRNA expression. The idea of employing the non-negative 
constrains is that miRNAs are assumed to down regulate their corresponding 
mRNA targets. Given the linear regression model, we could employ various 
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Figure 1: The models for mRNA and miRNA interactions: (A) miRNA directly 
targets mRNA and (B) the complex RISC composed of miRNA and Argonaute 
proteins targets mRNA. 



algorithms for resolving the model. For this study we have applied the Least 
Square Regression (LSR), Ridge Regression (RR), Least Absolute Shrinkage 
and Selection Operator (LASSO), non- negative LASSO (nLASSO), Bayesian 
LASSO (BLASSO) and non-negative Bayesian LASSO (nBLASSO) to the ex- 
perimentally validated interactions of four expression data sets. For the point 
estimate algorithms, we have observed that LSR, RR and LASSO produced the 
same results and while the nLASSO was able to produce sparse representations 
for the mRNA-miRNA interactions. The Bayesian approaches of BLASSO and 
nBLASSO output similar results. Since both the BLASSO and nBLASSO are 
not able to select variables automatically, we propose to calculate the credible 
intervals to take into account the prediction uncertainties for the variable se- 
lections. The statistical significance is calculated for the credible intervals and 
then the variables are selected based on this statistical significance given a sig- 
nificance probability a. Thus the numbers of selected variables are dependent on 
the given significance probability. We concluded that the Bayesian approaches 
produce more meaningful results, which can measure the uncertainty of the pre- 
dictions, than the point estimate based algorithms which only produce a single 
point for the parameters. 



2 The Model 

Given a set of expression data of miRNA and mRNA, we model the interactions 
between mRNA and miRNA by the following mathematical model 

Y ^ f (X) + E 
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where Y G TZ^^^ is the collections of the mRNA expression data where A'' is 
the number of tissue samples and P is the number of mRNAs, f is the function 
for modelling the interactions between Y and X, X S JZ^^^ is the collections 
of the miRNA expression data where M is the number of miRNAs, and E is 
the noise matrix. The model is very rich in presenting the miRNA targets and 
the function f could be non-linear. However in this paper we only explore the 
linear model which could then be represented as 

Y = Xf + E 



where now f e j^mxp matrix. This model is similar to the bilinear model 
proposed in Zhong et al. (2011), where both the X and f could be unknown 
or X is known and f is unknown. In this paper, since both the miRNA and 
mRNA expression data are observed, we will consider the later case. An illus- 
trate plot for the model is shown in the Figure [T] where the plot (A) models 
the direct interactions of mRNA and miRNA and (B) models the interactions 
of mRNA and RISC which is a complex composed of miRNA and Argonaute 
protein. Stanhope et al. (20091 and Lu et al. (2011) have studied the model 



(B) and shown the advantages of this model comparing to the model (A). We 
will consider the both models. So in this paper, the matrix X represents the 
miRNA for model (A) or the complex RISC for model (B). Note that X may be 
the multiplication of —1 and the miRNA expressions and thus a positive value 
of fmp represents the negative regulation of miRNA. Research has found that 
miRNA usually takes part in negative regulation in translational repression or 
target degradation and gene silencing (Bartel 2009 Kusenda et al. 20061, and 



however miRNA may also possibly act as positive regulation such as the tran- 
scriptional and translational activations (Vasudevan et al. 2007). So in order 



to model the roles of miRNA, we assume two situations /„„ > and f £TZ. 



2.1 Modelling both the positive and negative regulations 

As was noted we could assume that miRNAs act as both positive and negative 
regulations to their targets. Then we consider one gene denoted by y € TZ^^^, 
and P G j^Mxi column of f. The model is then represented as 

y = X/3 + 6 

where e is one column of the noise matrix E. We assume that e ^ A/'(0, cr^I) 
is Gaussian noise with zero mean and variances a^. The main task is then 
to estimate the parameter vector f3 which models the miRNA targets. Several 
algorithms are available for resolving this problem, such as the Least Square 
Regression, Ridge Regression, The Least Absolute Shrinkage and Selection Op- 
erator (LASSO) and Bayesian LASSO. 
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2.1.1 Least Square Regression 

The LSR for the Unear model is defined by the following optimization problem 



The solution has the following closed form 

/3 = (X^X)-iX^y 

2.1.2 Ridge Regression 

The ridge regression for the linear model is defined by, 
/3 = argmin l^lyi^Yl ^■J'"'-? 

subject to 

3 

The solution could be represented as 

/3= (X^X + AI)-iX^y 
where A is a regulation parameter. 

2.1.3 LASSO 

The LASSO estimate for the linear regression model is defined by, 

/3 = argmin |^ - ^ /3j^,j 
subject to 

We can see that no constrains are imposed on LSR, the RR uses a Z2-norm 
constrain on the /3 and while LASSO employs the Zi-norm. A few softwares are 
available for resolving LASSO and here we employed the Glmnct for Matlab 
(iFriedman et all 120071 12010|). 
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2.1.4 The Bayesian LASSO 

While LSR, RR and LASSO are point estimate algorithms, we also examine 
the Bayesian LASSO approach of Park and Casella (2008) which infers the 
posterior densities for the parameters /3. Given the linear regression model, the 
data likelihood follows a Normal distribution with the following form 

p(y|X,/3,a2)=AA(X/3,a2I^^) 

We employ the following priors for those parameters 

p{P\a\ tI ■■■ ,tIj)^N (Om, ^I'i^r) , 

where Dr — diag{Tf, • • • , r^^) 

p{X) cx (A2)'-ie-T 

So given the likelihood and priors, we are interested in simulating the conditional 
distribution 

p{f3, <J-\r-^, A2|y, X) (X p(y|X, /3, a')p{(3\a^, r^)p{T'\X)p{a^)p{X) 
The posterior distributions are then 

p(/3|X,y)=AA(M0,I]0) 
where iip = A'^X'^y, = a^A-\ and A = X'^X + D-\ 
p{<7^'^\X.,y) ^ Gamm,a{aa, Pa) 

where aa - ^{N+M)+2 and (S^ = + i(y - X/3)^(y - X^)] "\ and 

Gamma represents a Gamma distribution with the density f{x) = r(a);3° x°~"'"e~? . 

p{t^'^ \ ■ ■ ■) ^ InvGaussian{ii^ , X^) 



where fir — y^i^ and Xt — X^ , and InvGaussian represents an inverse Gaus- 
sian distribution which has a density given by f{x) = \J~^^~^ ^^P {^^ ^^tfJx } 
where a; > 0. 

p(A^| • • • ) ^ Gamma[a\, (3\) 

where ax = M+r and Px— (^5 + 5 Y^'jLi ■ K can be seen that all the pos- 

teriors are the known distributions and thus we could simulate the parameters 
by using the Gibbs sampler in the Algorithm 1. 
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Algorithm 1 Gibbs Sampler for Bayesian LASSO 
Require: y, X, r = 1, (5 = 1, NSamps 
for t=l:NSamps do 



Draw sample t~ ^ InvGaussian(iiT- , A^) for all j 
Draw sample /3 ^ A/'(/i^, S/?) 
Draw sample a^^ ^ Gamma{ac, (3^^) 
Draw sample ^ Gamma(a\, l3\) 
end for 



2.2 Modelling the negative regulations 

The negative interactions of miRNA and mRNA are modeled by the following 
linear regression model when /3 is non-negative, 

y = -X/3 + e 

where now X is the miRNA expression data. To resolve this model, we employ 
both the point estimate algorithm and the Bayesian solution. 

2.2.1 Non-negative LASSO Algorithm 

The nLASSO for the linear regression model is defined by 

/3 = ar grain S X! h^' + X! 



J 



subject to 

3 

We see that the only difference between LASSO and nLASSO is that nLASSO 
also employs the non-negative constrains. For implementing this algorithm, we 
employ the Matlab software available at ,http://www. stanford.edu/~boyd/ll-ls/] 



(Kim et al , 20071. 



2.2.2 Non-negative Bayesian LASSO 

Similar to the Bayesian LASSO, we could also simulate the negative regulators 
by using sampling approaches. To simulate the required posterior distribution, 
we employ the following data likelihood and the priors for those parameters 

A) = n ^ exp |-^| > 0) 
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p{a^) cx 



1 

We are interested in simulating the conditional distribution 

p(/3, a^, A|y, X) cx p(y|X, f3, a^)p{f3\a^ , \)p{a^)p{X) 

The conditional distribution for f3 is then a truncated Normal 

p(/3|y,X)cxAA+(Ai/j,S/3),/3, >0 

where /it^ = -I]x(X^y + ^A) , = cr^Sx where Sx = (X^X)-\ and A = 
(Ai, A2, • • • , Am)"^- Sampling from the truncated Normal distribution is shown 
in the Appendix. The conditional distribution for cr^^ ^ Gamma^aa, f3a-) is 



a Gamma distribution where P„ 



i(y + X/3)^(y + X/3) + i E™=i A™/3„ 
and Q!o- = + Af + 2. The conditional for Am is also a Gamma distribution, 

i.e. A„i Gamma{ax, /3a), where = Q;° + 1 and = (;go- + /3m) • So 
finally we use the Gibbs sampler in Algorithm 2 for simulating those parameters. 

Algorithm 2 Gibbs Sampler for Non-negative Bayesian LASSO 
Require: y, X, a° = le — 6, = le6, NSamps 
for t=l:NSamps do 

Draw sample /3 ^ Af+{fJ,p, S/3) 
Draw sample cr^^ ^ Gamma{a„, (i^) 
Draw sample Am ~ Gamma{a\, Px) for m = 1, 2, • • • , M. 
end for 



3 A Variable Selection Scheme 

Both the Bayesian Lasso and the non-negative Bayesian Lasso are not able to 
select variables automatically. Here we propose a particular scheme for variable 
selection for this problem. In Bayesian statistics, credible interval is usually used 
to summary the statistics for the inferred posterior distribution of a parameter 



(See for example Eberly and Casella (2003)). We define a credible interval [a, 6] 



for a variable. If a is significantly greater than zero, we assume that the variable 
is active (or selected). So we call it the active credible interval (ACI). Since many 
of the posterior distributions are multimodal, we partition the posterior samples 
into two clusters. We observed that for this particular problem the posterior of 
P has high probability density around zero and also it may have high probability 
density apart from zero. So if the posterior really has high probability density 
greater than zero, we would assume the corresponding variable is active. Ideally, 
the samples of the posterior are partitioned into two clusters, where one cluster 
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has high probabihty density around zero and the other is apart from zero. We 
calculated the ACL The statistical significance of the ACI is then defined as 



where Q is the number of samples in the ACI and T is the totally number of 
the samples. For fixing a particular value < a < 1, if p > a, we then assume 
that the variable is selected (or active). Obviously, the value of a controls the 
numbers of active variables. As the value of a increasing, the smaller numbers 
of variables are selected. 



4 Expression Data Sets 



The experimentally validated targets of the four data sets were employed to val- 
uate those linear regression algorithms. The first data, which is noted as Multi 
Class Cancer (MCC), has been studied by |Muniategui et al. (2012a). In this 



data, 23 mRNA-miRNA interactions have been identified by searching from the 
literatures. This data set has 88 paired cancer and normal samples with mRNA 



data from Ramaswamy et al. (2001 ) and miRNA expression data from |Lu et al. 
(2005). We employed the experimentally validated targets reported in Muni- 
ategui et al. (2012a') and the method there for selecting the initial set of putative 



interactions. The following three data sets, which are also using Argonaute pro 
tein expression, were studied by Lu et al. (2011) and Stanhope et al. (2009) 



The second data was to study the nasopharyngeal cancer by those researchers 
in Madison, WI, USA which derived mRNA and miRNA profiling from 31 NPC 
and 10 normal tissule samples. The third data set was from the Broad institute 
which contains 67 tissue samples of 10 different normal and tumor tissue types. 
The fourth data set was conducted by Memorial Sloan-Kettering Cancer Center 
(MSKCC) with 28 normal, 98 primary cancer and 13 metastatic cancer tissues 
samples using the Affymetrix Human Exon 1.0 ST Array and Agilent Human 
miRNA Microarray 2.0. We use the experimentally detected mRNA-miRNA in- 



teractions for the data sets Broad, Madison and MSKCC collected by (Lu et al 



20111 to compare those algorithms. The numbers of experimentally validated 



interactions are listed in the Table [7] for all the data sets. 



5 Results 

In this section we apply six linear regression algorithms to four expression data 
sets. The six linear regression algorithms are least square regression, ridge 
regression, LASSO, non-negative LASSO, Bayesian LASSO and non-negative 
Bayesian LASSO. The first four algorithms produce point estimates for the 
model and the last two methods are Bayesian methods. 
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5.1 Bayesian Methods Perform Better in Modelling Sparse 
Interactions for mRNA and miRNA 

We have applied the six hnear regression algorithms to the known targets of 
the four data sets described. The four algorithms, i.e. LSR, ridge regression, 
LASSO and non-negative LASSO, produce point estimates for those parame- 
ters. Across all the data sets employed, the LSR, ridge regression and LASSO 
produced the same results. We observed that across all the data sets, LSR, 
ridge regression and LASSO algorithms were not able to learn sparsity for the 
model (see for example Figures |2] & |4| and thus additional schemes such as the 



one proposed in Lu et al. ( j2011j have to be empl oyed for the variable selection 
purposes. As was reported in [Muniategui et al. (2012a), we also confirm that 



the nLASSO was able to learn model sparsity. See the Figures [2] & |4] Figure 
[2] shows the learned /3 for the gene TWISTl where there are 21 candidate mi- 
croRNAs. Looking at this example, the non-negative LASSO algorithm clearly 
identified three miRNAs and they are miR-137, miR-145 and miR-200c and all 
the coefhcients for the rest miRNAs tend to zero. However, the LSR, Ridge 
regression and LASSO estimated 10 negative values for /3 and 11 positive values 
and all those values were not tending to zero. Thus, 11 miRNAs were iden- 
tified for down regulating TWISTl. This shows that the algorithms with the 
non-negative constraint on the parameters is more suitable for modelling sparse 
interactions of mRNA and miRNA than those without any constraints, since we 
are mainly looking for miRNAs which down regulate mRNAs. Although LASSO 
algorithm is not considering model sparsity for this particular problem, it iden- 
tified more miRNAs than nLASSO. This may be an advantage for employing 
LASSO algorithm, comparing to the nLASSO algorithm, since it selects more 
miRNAs. Comparing to those point estimate algorithms, we can infer the pos- 
terior densities for those parameters for the gene TWISTl using the BLASSO 
and nBLASSO. Comparing to the LASSO algorithm, however, the Bayesian 
LASSO where no non-negative constraints were employed was also able to pro- 
duce convincible results for identifying mRNA-miRNA interactions. This may 
be due to that Bayesian LASSO was able to explore the whole parameter spaces, 
and while the LASSO is required to tune the regularizing parameter A which 
is crucial for learning the model sparsity. The non-negative Bayesian LASSO 
performs best among those algorithms employed. Figure [3] shows the inferred 
densities for five miRNAs. Looking at those densities, for the ones inferred by 
BLASSO, we would confidently select miR-145, miR-200b and miR-200c and ig- 
nore miR-141 and miR-200a since the densities of miR-141 and miR-200a seem 
symmetric around zero. While for the ones inferred by nBLASSO, we would 
confidently select miR-200b and miR-200c since the densities for them have two 
peaks at around both zero and 0.05 and we are not sure to select miR-141, 
miR-145 and miR-200a since they have only one peak at around zero. We then 
need to calculate the active credible intervals and their statistical significance 
to perform the variable selections. For the gene TWISTl, the results for the 
active credible intervals and their statistical significance are shown in the Ta- 
ble [1] for the Bayesian LASSO and the Table [2] for the non-negative Bayesian 
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LASSO. If we set the parameter a = 0.05, the BLASSO selected miR-145, 
miR-200b and miR-200c and while the nBLASSO selected miR-141, miR-145, 
miR-200a, miR-200b and miR-200c. It should be noted that the miR-200 family 
(miR-200a, miR-200b, miR-200c and miR-141) were experimentally identified 



as down-regulating TWIST 1 ( |Wiklund et al.[|2011[ ). We analyzed all the genes 
using the same procedure. 

We show all the credible intervals and their statistical significance which are 
greater than 0.05 for the experimentally validated genes of the MCC data set 
computed by BLASSO and nBLASSO in the Tables |6] & [§ respectively. We 
set a = 0.05 for selecting the miRNAs. Note that to perform variable selec- 
tion, we have to manually fix a threshold for the point estimate algorithms such 
as the LASSO and nLASSO. We see that nBLASSO selected more miRNAs 
than BLASSO, which is due to the non-negative constrains. We also counted 
the numbers of experimentally validated interactions estimated by the LASSO, 
nLASSO, BLASSO and nBLASSO for all the four data sets which are shown 
in the Table [Tj Note that the analysis for the three data sets Broad, Madi- 
son and MSKCC which are using Argonaute protein information is shown in 
the following sections. We see that the Bayesian approaches identified more 
miRNA and mRNA interactions than the point estimate algorithms across all 
the data sets, which conclude that the Bayesian approaches perform better than 
the point estimate algorithms in modelling the mRNA and miRNA interactions. 



5.2 Bayesian Analysis for mRNA-miRNA Interactions Us- 
ing Argonaute Protein Information 

We have shown that both the Bayesian approaches and the non-negative LASSO 
for linear regression model are able to identify the sparse representations for 
the miRNA-mRNA interactions. It has been shown that the regression models 
employing Argonaute expressions fit the expression profiles of known targets 
pairs substantially better than models based only on miRNA expression data 
(Stanhope et al. 2009 Lu et al. 2011). We applied all the six algorithms to 



three expression data sets, which are Broad, Madison and MSKCC, involving 



2009 


I and 


Lu et al. 


(2011 



Here we analyze the results for the gene NOTCHl 
and we analyze all the rest genes using the same method. Twelve miRNAs 
were selected as the candidates for interacting with the gene NOTCHl. Six 
miRNAs have been experimentally detected to down regulate NOTCHl and 
they are miRNA 34a, 34b & 34c, miRNA 23b, miRNA 24 and miRNA 27b 



(Fukuda et al. 2005 Wang et al. 2010 Zheng et al. 2012 Melo and Kalluri 



20121. The point estimate algorithms, i.e., the LSR, ridge regression, LASSO 
and non-negative LASSO, were applied to the expression data for identifying 
the interactions of the mRNA and miRNA. The estimated /? are shown in the 
Figure |4j In the figure, the black bars plot the /3 for the miRNA involving 
Argonaute 2 expression information and the red bars plot the /3 for the miRNA 
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involving Argonautc 1,3&4 expression information. Obviously the LSR, ridge 
regression and LASSO produce similar results. Since the Argonaute 2 protein is 
the competitor of Argonaute 1,3&4 in forming the RISC for targeting mRNA, 
as was discussed in Stanhope et al. (2009) the values of /3 for miRNA with Ago2 
information computed by LSR, ridge regression and LASSO are inversely related 
to that of miRNA with Agol,3,&4 information in sign with similar magnitudes 
of the (3 values. For the results of the non-negative LASSO, if the effect of a 
miRNA with Ago2 is positive, the effect of the miRNA with Agol,3&4 then is 
zero, and vice versa, and also both the effects may be zero. For the LSR, ridge 
regression and LASSO, we would prefer selecting all of the candidate miRNAs, 
and while the non-negative LASSO selects smaller number of miRNAs. We 
also applied the Bayesian approaches to this data. The probability densities are 
plotted in the Figures [S] & |6] for Bayesian LASSO and non-negative Bayesian 
LASSO, respectively. We then calculated the active credible intervals and as 
well as the statistical significance which are shown in the Tables |3] & |4] We 
use the parameter a = 0.05. The Bayesian LASSO selected seven miRNAs 
combined with Ago2 and five miRNAs combined with Agol,3&4 which contains 
5 known targets out of 6 and the nBLASSO selected eight miRNAs combined 
with Ago2 and seven miRNAs combined with Agol,3&4 which contains 5 out 
of 6 known targets. Their densities are shown in Figures [5] & [6j For the purpose 
of clearly comparing the effects of Ago2 and Agol,3&4, we show the results 
for gene ACAA2 where only the miRNA-124 was selected as the candidate 
for targeting ACAA2. Figure [7] plots the traces for sampling the effects of 
miRNA- 1 24 xAgo2 and miRNA-124x Agol,3&4. Interestingly, when the effects 
of miRNA- 1 24 xAgo2 are on, the effects of miRNA-124x Agol,3&4 are then off 
and vice versa. This implies that Ago2 and Agol,3&4 are competing with each 



other to form the RISC which agrees the results of Stanhope et al. ( 2009 ) 



6 Conclusions 

As linear regression models have been proved suitable for modelling miRNA 
targets, we have proposed Bayesian LASSO and non-negative Bayesian LASSO 
for inferring mRNA-miRNA interactions. We have applied the least square re- 
gression, ridge regression, LASSO, non-negative LASSO, Bayesian LASSO and 
non-negative Bayesian LASSO to four publicly available expression data sets. 
We conclude that LSR, RR and LASSO algorithms perform the same, and 
the nLASSO produces sparse representations for the mRNA-miRNA interac- 
tions since the non-negative constrains were imposed. However we argue that 
LSR, RR and LASSO estimate more numbers of mRNA-miRNA interactions 
than nLASSO. Both the BLASSO and nBLASSO provide uncertainties for the 
estimates of the interactions. The Bayesian methods do not perform variable se- 
lections automatically and thus we proposed to employ the credible intervals to 
accomplish this task by looking at a statistical significance. Of course, the pint 
estimate algorithms are also lack of variable selection automatically and thus a 
threshold may have to be manually fixed for selecting miRNAs. However, we 
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could employ advanced methodologies for performing variable selections for this 
problem as was proposed in Zhong et al. (2011). We prefer the Bayesian ap- 
proaches rather than the point estimate algorithms, because Bayesian methods 
produce the probability densities for the effects of the miRNAs to their targets. 
So we can analyze the mRNA-miRNA interactions by graphing those inferred 
densities of the effects. It has been well known that the Ago2 proteins are 
competitors of the Agol,3&4 proteins in binding with RISC. Our Gibbs sam- 
pler was able to simulate this competition by sampling the effects of the RISC 
to miRNA targets. Overall, comparing to point estimate algorithms, Bayesian 
methods were able to infer more numbers of known mRNA and miRNA inter- 
actions. 



Appendix 

In order to simulate the truncated Normal distribution, we employ a Gibbs 
sampler. We consider sampling from the general truncated Normal distribution 
of the form 

p(z|/i, S) « exp (-^(z - m)^S-1(z - fi)^ I(z e C) 

We are to sample Zi given z_i = {zi,Z2,--- , z^+i, • • • ,zm)- The condi- 
tional distribution is then 

p{zi\z_i) oc exp (^-^{^i -""0^^ li^i ^ {ci,di)) 

where = ijJ~i^ , Ui = + ^j^iil^j ~ where uJij is the ijth element of 

Set 5 = it is required to simulate p(^) oc exp(-^^/2)/(^ e (cj,dj)) 

where = and — . We employ the Gibbs sampler in 



Damien 



and Walker (20011 to simulate p{(,)- Then ^ is simulated in turn accordmg to 



the conditionals which are Uniform: 

p(r|0^t/(0,exp(-eV2)) 
Pi^\y) U (max (c^, -V-21ogy) , min (d^, 2 logy 
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Figure 2: The point estimates for {3 for the gene TWISTl in the MCC data 
set using Least Square Regression, Ridge Regression, LASSO and Non-negative 
LASSO algorithms. 



mRNA 


miRNA 


Significance 


Credible Interval 


TWISTl 


miR-145 


0.104 


[0.018,0.068] 


miR-200b 


0.153 


[0.028,0.083] 


miR-200c 


0.691 


[0.032,0.086] 



Table 1: The active credible intervals and their statistical significance computed 
by Bayesian LASSO for gene TWISTl {a = 0.05). The miRNAs highlighted by 
red were experimentally validated targeting TWISTl. 



mRNA 


miRNA 


Significance 


Credible Interval 


TWISTl 


miR-141 


0.073 


[0.023,0.113] 




miR-145 


0.104 


[0.026,0.132] 




miR-200a 


0.058 


[0.023,0.118] 




miR-200b 


0.193 


[0.028,0.111] 




miR-200c 


0.390 


[0.028,0.096] 



Table 2: The active credible intervals and their statistical significance computed 
by non-negative Bayesian LASSO for gene TWISTl (a = 0.05). The miRNAs 
highlighted by red were experimentally validated targeting TWISTl. 
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Figure 3: The inferred probability densities for /3 by using the Bayesian LASSO 
(left) and non- negative Bayesian LASSO (right). The miRNAs highlighted by 
red were experimentally validated as down regulating the gene TWISTl. 



niRNA 


miRNA 


Significance 


Credible Interval 


NOTCHl 


miR-92xAgo2 


0.058 


[0.048,0.217] 




miR-139xAgo2 


0.056 


[0.054,0.171] 




niiR-30dxAgo2 


0.162 


[0.071,0.198] 




miR-34cxAgo2 


0.262 


[0.041,0.138] 




miR-23bxAgo2 


0.182 


[0.064,0.203] 




miR-24xAgo2 


0.234 


[0.067,0.220] 




miR-27bxAgo2 


0.066 


[0.039,0.154] 




miR-92xAgol34 


0.078 


[0.024,0.086] 




miR-139xAgol34 


0.154 


[0.046,0.167] 




miR-30dxAgol34 


0.530 


[0.071,0.191] 




miR-34bxAgol34 


0.118 


[0.047,0.146] 




miR-24xAgol34 


0.138 


[0.051,0.207] 



Table 3: The inferred active credible intervals and their statistical significance 
computed by Bayesian LASSO {a — 0.05). The miRNAs highlighted by red 
were experimentally validated to down regulate NOTCHl. 
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Figure 4: The point estimates for /3 using Least Square Regression, Ridge Re- 
gression, LASSO and Non-negative LASSO algorithms for gene NOTCHl using 
Argonaute expression, where black bar is the values for miRNA using Argonaute 
2 and while the red is using Argonaute 1,3, and 4. 
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Figure 5: The probability densities for /3 inferred by using the Bayesian LASSO 
for the gene NOTCHl, where miRNA-23b, miRNA-24, miRNA-27b, miRNA- 
34b and miRNA-34c were experimentally validated to down regulate gene 
NOTCHl. 
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Figure 6: The probability densities for /3 inferred by using the non- negative 
Bayesian LASSO for the gene NOTCHl, where miRNA-23b, miRNA-24, 
miRNA-27b, niiRNA-34b and miRNA-34c were experimentally validated to 
down regulate gene NOTCHl. 
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Figure 7: The samples for (3 drawn by using Bayesian LASSO and non-negative 
Bayesian LASSO, where the black plots the /3 for iniRNA-124x Ago2 and the 
red plots for miRNA-124x Agol34. 



niRNA 


miRNA 


Significance 


Credible Interval 


NOTCHl 


miR-92xAgo2 


0.124 


[0.046,0.156] 




miR-139xAgo2 


0.175 


[0.061,0.203] 




niiR-30dxAgo2 


0.249 


[0.059,0.177] 




miR-34cx Ago2 


0.102 


[0.033,0.118] 




miR-34bxAgo2 


0.054 


[0.043,0.158] 




miR-23bxAgo2 


0.098 


[0.036,0.136] 




miR-24xAgo2 


0.191 


[0.051,0.168] 




miR-27bxAgo2 


0.051 


[0.022,0.077] 




miR-92xAgol34 


0.093 


[0.028,0.109] 




miR-139xAgol34 


0.121 


[0.042,0.146] 




miR-30dxAgol34 


0.262 


[0.047,0.145] 




miR-34cxAgol34 


0.121 


[0.026,0.097] 




miR-34bxAgol34 


0.071 


[0.032,0.137] 




miR-23bxAgol34 


0.081 


[0.021,0.083] 




miR-24xAgol34 


0.143 


[0.038,0.117] 



Table 4: The inferred active credible intervals and their statistical significance 
computed by non-negative Bayesian LASSO (a = 0.05). The miRNAs high- 
lighted by red were experimentally validated to down regulate NOTCHl. 
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mRNA 


miRNA 


Significance 


Credible Interval 


mRNA 


mlRNA 


Significance 


Credible Interval 


EEF1A2 


let-7f 


0.628 


[0.107,0.239] 


CDKN2A 


miR-125b 


0.155 


10.051, 0.197) 


miR-181b 


0.067 


[0.046, 0.178) 


miR-195 


0.057 


10.033, 0.151] 


miR-182 


0.066 


[0.037, 0.151) 


miR-29a 


0.051 


10.066, 0.305] 


FSCNl 


miR-133a 


0.458 


[0.071, 0.201) 


miR-99a 


0.085 


10.032, 0.135) 


miR-145 


0.103 


[0.041, 0.148) 


CEACAM5 


mir-143 


0.079 


10.058, 0.223) 


miR-29c 


0.291 


[0.084, 0.262) 


miR-145 


0.096 


10.068, 0.254) 


BRAF 


lel,-7f 


0.053 


[0.037, 0.154) 


miR-221 


0.057 


10.071, 0.283) 


miR-145 


0.150 


[0.041, 0.143) 


KRT14 


miR-143 


0.054 


10.029, 0.123) 


miR-183 


0.059 


[0.038, 0.157) 


IGFBP6 


miR-181b 


0.115 


[0.051, 0.182) 


miR-192 


0.084 


[0.016, 0.063] 


miR-27a 


0.073 


10.048, 0.185) 


miR-221 


0.103 


[0.049, 0.188) 


PIGR 


miR-128b 


0.279 


10.048, 0.152] 


miR-223 


0.318 


[0.063, 0.200] 


PLKl 


lel,-7b 


0.314 


[0.125, 0.399) 


miR-9 


0.107 


[0.044, 0.167) 


miR-100 


0.411 


10.046, 0.136] 


CCXDl 


miR-i:)Ob 


0.171 


[0.176, 0.702] 


miR-99a 


0.129 


10.031, 0.107] 


miR-194 


0.097 


[0.031, 0.138] 


miR-99b 


0.164 


10.075, 0.247] 


miR-7 


0.129 


[0.061, 0.270) 


BAKl 


miR-128b 


0.431 


10.038, 0.109] 


HOXDIO 


let-7b 


0.101 


[0.124, 0.595) 


miR-16 


0.308 


[0.053, 0.164] 


miR-103 


0.050 


[0.056, 0.277) 


miR-25 


0.117 


[0.038, 0.137) 


miR-125b 


0.088 


10.041, 0.203) 


miR-29a 


0.052 


[0.047, 0.194] 


miR-141 


0.052 


10.026, 0.139) 


miR-32 


0.088 


10.032, 0.119) 


miR-145 


0.072 


10.038, 0.189] 


E2F1 


miR-16 


0.059 


10.051, 0.302] 


miR-15b 


0.067 


10.048, 0.230] 


miR-19S 


0.173 


10.032, 0.145] 


miH-200b 


0.082 


10.028, 0.134) 


miR-23b 


0.097 


[0.044, 0.234) 


miR-200c 


0.125 


10.028, 0.131) 


miR-29a 


0.089 


[0.068, 0.355) 


miR-21 


0.054 


10.047, 0.254) 


TWISTl 


miR-141 


0.073 


10.023, 0.113] 


miR-25 


0.078 


10.061, 0.302) 


miR-145 


0.104 


[0.026, 0.132) 


niiR-7 


0.078 


0,035. 0.1821 


miR-200a 


0.058 


[0.023, 0.118) 


CDKX_'A 


iiiiiM 


1) 2:il 


(l.l)2!>. 102 


miR-200b 


0.193 


[0.028. 0.111) 


imll-illU 


11-0^7 


0,1111. Il,l(i>)| 


iuiR-200c 


0.390 


[0.028, 0.096) 



Table 5: The inferred statistical significance and active credible intervals us- 
ing non-negative Baycsian LASSO for the MCC data. Only the miRNAs with 
significance greater than 0.05 are shown, where the bolded miRNAs are exper- 
imentally validated for repressing their targets. 
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niRNA 


miRXA 


Sij^iiificfliicp 


Credible Interval 


mRNA 


miRNA 


Significance 


Credible Interval 


EEF1A2 


let-7f 


0.631 


[0.098,0.239] 


HOXDIO 


miR-7 


0.084 


[0.026, 0.098] 


FSCNl 


miR-133a 


0.387 


[0.067, 0.201] 


CDKN2A 


miR-1 


0.501 


[0.039, 0.109] 


miR-145 


0.088 


[0.044, 0.175] 


miR-100 


0.077 


[0.038, 0.137] 


miR-29c 


0,252 


[0.095, 0.331] 


miR-125b 


0.131 


[0.048, 0.165J 


BRAF 


miR-145 


0.120 


[0.040, 0.138] 


miR-195 


0.069 


[0.038, 0.144J 


miR-192 


0.083 


[0.015, 0.058] 


miR-99a 


0.104 


[0.041, 0.141] 


miR-221 


0.072 


[0.043, 0.159] 


CEACAM5 


miR-143 


0.072 


[0.062, 0.237] 


miR-22:) 


0.337 


[0.063, 0.194] 


miR-145 


0.073 


[0.068, 0.258] 


miR-9 


0.086 


[0.041, 0.151] 


miR-221 


0.074 


[0.086, 0.323] 


CCNDl 


miR-1 


0.181 


[0.034, 0.117] 


IGFBF6 


miR-181b 


0.079 


[0.047, 0.176] 


miR-130a 


0.100 


[0.063, 0.238] 


PIGR 


miR-125b 


0.238 


[0.051, 0.170] 


miR-130b 


0.126 


[0.142, 0.490] 


PLKl 


let- 7b 


0.213 


[0.101, 0.336] 


miR-194 


0.211 


[0.033, 0.109] 


miR-100 


0.533 


[0.048, 0.132] 


miR-7 


0.166 


[0.054, 0.180] 


miR-99a 


0.161 


[0.076, 0.239] 


HOXDIO 


let-7b 


0.075 


[0.097 


0.362] 


miR-99b 


0.161 


[0.076, 0.239] 


miR-103 


0.080 


[0.0.58 


0.216] 


BAKl 


miR-125b 


0.467 


[0.037, 0.106] 


leH07 


0.064 


[0.0-54 


0.205] 


miR-16 


0.378 


[0.0-56, 0.168] 


miR-125b 


0.110 


[0.036, 0.128] 


miR-25 


0.087 


[0.035, 0.129] 


miR-141 


0.071 


[0.024, 0.092] 


miR-32 


0.070 


[0.029, 0.112] 


miR-145 


0.087 


[0.034, 0.122] 


E2F1 


miR-16 


0.057 


[0.042, 0.160] 


miR-15b 


0.093 


[0.050, 0.179] 


miR-195 


0.58 


[0.048, 0.129] 


miR-195 


0.050 


[0.029, 0.118] 


miR-23b 


0.091 


[0.039, 0.140] 


miR-200b 


0.098 


[0.026, 0.095] 


miR-29a 


0.067 


[0.051, 0.190] 


miR-200c 


0.219 


[0.029, 0.099] 


TWISTl 


miR-200c 


0.691 


[0.032, 0.086] 


miR-21 


0.057 


[0.034, 0.130] 


miR-145 


0.104 


10.018, 0-068] 


miR-25 


0.099 


[0.065, 0.232] 




miR-200b 


0.153 


[0.028, 0.083] 



Table 6: The inferred statistical significance and active credible intervals using 
Bayesian LASSO for the MCC data. Only the miRNAs with significance greater 
than 0.05 arc shown, where the bolded miRNAs are experimentally validated 
for repressing their targets. 



Data 


Nos of Validated Targets 


LASSO 


nLASSO 


BLASSO 


nBLASSO 


MCC 


23 


17 


13 


16 


20 


Broad 


76 


34 


43 


47 


47 


Madison 


99 


37 


52 


60 


84 


MSKCC 


106 


52 


52 


58 


59 



Table 7: The numbers of experimentally validated interactions estimated by 
LASSO, nLASSO, BLASSO and nBLASSO. 
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